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Several theorems are proved concerning the solutions of equations that 
arise in the study of resistive nonlinear electrical networks. The first, an 
existence and uniqueness theorem, applies to equations describing an 
interesting class of networks which includes certain active and nonreciprocal 
networks for which the existence and uniqueness of solutions has not 
previously been established. A method of computing bounds on the location 
of the solutions is given, and two iterative techniques are presented for 
computing the solutions. It is proved that the iterative techniques converge 
for a subclass of the equations which also includes equations describing 
certain active and nonreciprocal networks. Finally, the rate of convergence 
of the iterative techniques is compared with that of another well-known 
iterative technique and some practical computational aspects are pointed 
out. Computations for two example problems, not reported here, were carried 
out to show the practicality of applying these iterative techniques to the 
equations of specific networks. 

I. INTRODUCTION 

In this paper we consider the solution of the equation 

F(x) + Ax = B (1) 

where x = ■ is a point in the ??-dimensional Euclidean space E n , 



F(x) - 



Ux n ) 



is a nonlinear function mapping E" into E", A is an 
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n X n matrix of real numbers, and B m • is an arbitrary point 

.fr- 
ill ET. We prove (Theorem 1) that there is a unique solution of (1) if: 
(i) Each /,• is a strictly monotone increasing function mapping 
E 1 onto E 1 , 
and 

(u) The elements a,, of the matrix A satisfy the inequality 

a,, > 23 I a u 1 1 f° r * =* If ■ ■ • t »■ 

We then demonstrate a straightforward method of computing bounds 
on the location of this solution. Finally, we present two iterative tech- 
niques for computing the solution; and prove (Theorem 3) that the 
two additional assumptions: 

(in) Either all of the functions /,• are convex, or else all /< are concave, 
and 

(iv) a tt ^ if i 9* j, 
are sufficient to guarantee that the iterations converge to the solution. 

Equations of type (1) occur often in the study of nonlinear electrical 
networks. For example, if a linear n-port containing resistors, inde- 
pendent sources, and dependent sources has a two-terminal device 
whose V vs I curve is specified by J,- = /<(V<), for i = 1, • • • , n, con- 
nected across each port, then the port voltages may often be expressed 
as the solution of an equation of type (1). In this case the matrix A 
will be the ^/-parameter matrix of the n-port, the constant vector B 
will account for the presence of the independent sources, and the com- 
ponents of the vector x will be the desired port voltages. 

II. ACTIVE AND NONRECIPROCAL n-PORTS 

In case the n-port of the above example contains no dependent 
sources and the functions /< satisfy condition (i) above, the existence 
and uniqueness of a solution of (1) follows immediately from the 
well-known result of R. J. Duffin. 1 In fact, with the additional as- 
sumption that the slope of each U is bounded by positive constants 
the computational technique of J. Katzenelson and L. H. Seitelman 
may be used to compute the solution. 2 This computational technique 
is based upon a theorem of I. W. Sandberg which relies upon the 
contraction-mapping fixed point theorem. 3 
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Sandberg's theorem may, in fact, be used to prove the existence 
and uniqueness of a solution of (1), and to construct a convergent 
iteration process for computing this solution, whenever the matrix 
A is positive semidefinite* and the slope of each U is bounded by 
positive constants. Other theorems which do not require that the 
slopes of each strictly monotone increasing /< be bounded by positive 
constants also exist. (For example, see Ref. 4.) These theorems guar- 
antee existence and uniqueness of a solution of (1) whenever A is 
positive semidefinite but do not specify a procedure for computing it. 

Suppose, however, that the matrix A is not positive semidefinite; 
that is, suppose the n-port in the above example is active. Then the 
above results no longer apply. It may often happen that the matrix 
A is not positive semidefinite but still satisfies condition (ii) above. 
The matrix 



A = 



1 1 
5 7. 



for example, has this property. It is interesting to notice that in this 
case the matrix A will necessarily also be nonsymmetric (the corre- 
sponding n-port will be nonreciprocal) . This follows from the fact 
that for symmetric matries A, condition (ii) implies that A is a 
dominant matrix 5 which, in turn, implies that A is positive semi- 
definite. It is for this class of active nonreciprocal n-ports that our 
work provides entirely new results. Even for the passive case, how- 
ever, notice that our computational techniques do not require that 
the slopes of the functions /< be bounded. Also, there is reason to 
believe that for certain problems our iteration schemes may converge 
more rapidly than the ones based upon the contraction mapping 
theorem. More is said about this in Section VII. 

III. EXISTENCE AND UNIQUENESS 

Before proving the existence and uniqueness theorem we first prove 
a lemma which is used many times in this and the following section. 
Lemma 1: Let the n X n matrix A of real numbers satisfy condition 
(ii) of Section I. For j = 1, • • • , n let p { denote the jth component of 
p c E n . Let k t [1, • • •, n\ be chosen such that \p k \ = max {\p t \ : j = 
1, • • • , n). Then, 



* The n X n matrix A is said to be positive semidefinite if (x, Ax) > for 
I x in E n . 



all x in E\ 



1758 THE BELL SYSTEM TECHNICAL JOURNAL, OCTOBER IOCS 



and 



Proof: 



Pk < o => 2 o«p» s= °- 



a** 



n n •• 

Pk I ^ E I «*» I- 1 P* I ^ El a «P* I ^ E a*/P/ 



f-1 

IV* 



J-l 



i-l 
»Vfc 



Thus, 



a kk | p fc | ^ ±2 OmPi . 



;=i 



But then, 

p* > => a«p* ^ - E a *'P> =* E ««P* ^ 0. 






and, 



p* < => — OuP* ^ E a*/Pf «=* E a "iPi ^0- □ 



i=i 



Theorem 1 : There exists a unique solution of (1) whenever conditions 
(i) and (ii) of Section I are satisfied. 

Proof: We first prove that if a solution exists it is unique. Let x 1 
and x 2 be solutions of (1) . Then, 

Fix 2 ) - Fix 1 ) = A(x l - x 2 ). 

For j = 1, • • • , n let x) and x 2 denote the jth components of x 1 and x 2 , 
respectively, and choose k t {1, ■ • ■ , n} such that 

\xl - x 2 k \ = max {\x) - x 2 \ :j = 1, ■■■ ,n}. 

If x\ > x\ then, by Lemma 1, 

U(xl) - Uxl) = E a ki (x) - x 2 ) ^ 0. 

7=1 

If x\ < x\ then, by Lemma 1, 

Uixl) - Uxl) = E *M - *£) ^ 0- 
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Both of these conclusions contradict the fact that j k is strictly monotone 
increasing. Thus, x\ = x\ and hence x) = x* for j = 1, • • • , n. That is, 
the solution of (1) is unique, if it exists. 

We prove existence of a solution by induction. For k = 1, • • • , n let 



F k (x) B 



/.(*.) 



./*(**) 



A k = 



B, = 



.6*. 



Clearly, the matrix A k satisfies condition (ii) of Section I. Also, it is 
clear that there exists a unique solution* of F x (x) + A t x = B t for 
every strictly monotone increasing function / x mapping E 1 onto E 1 . 
Assume that there exists a solution of F k (x) + A k x = B k for arbi- 
trary strictly monotone increasing functions /,-, i = 1, ■ • • , k mapping 
E l onto E 1 . Then, for every real number x k+l , the equation 



F k (x) + A k x + 



■&k.k + \ . 



£*+i — B k 



has a (unique) solution; since for i = 1, • • • , k the function /<(**) + 
a-i.k+\X k+ x is strictly monotone increasing from E 1 onto E 1 . Let the 
components of this solution be denoted by x, = m t (x k+ i) fori = 1, • • • , k. 
We have thus denned k functions m,- on E 1 . 
We now prove that for every x\ +] , x 2 k+i t E 1 , 

I &* +] - x\+i | ^ | w,(.rLi) - ro,(a£+i) |, for j - 1, • • • , A;. (2) 

This inequality, incidentally, implies that each m,- is continuous. 
Let xi +1 , xl +i t E 1 and choose I e {1, • • • , &} such that 



W/(.T* + i) — »it(a£ + i) | 

= max {I 7*1,(3*+,) — w ; (.t1. +1 ) I : ;' = 1, 



,k\ 



Assume that | mi(x 2 k+1 ) — nit(xl +} ) | > | x 2 k+l — x\ +l | . Clearly, then, 
m.i(x\ + ,) - m t (xl +1 ) ^ 0. Ifm,(a:t + ,) - m,(xi +I ) > then, 

f,[ro,(a£ +1 )] - fi[f»i(a4 + ,)] > 0. 



* We take the liberty of using the same symbol i to denote points in any of the 
spaces E k , \ < k < n. No confusion should arise since the subscripts on F and A 
will make our choice clear. 
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Also, since the matrix A k+1 satisfies condition (u) of Section I, letting 
Pi = mi(x* k+l ) - m,-(zj +1 ), for j = 1, ■■• ,k, 

2 1 

Pi + 1 — X k + \ ~ X k + \ • 

■we have, by Lemma 1, 

E a»N(xli) - rmixLi)] + a t . k+l (x 2 k+i - x k+l ) ^ 0. 
j-i 

Thus, 

k 

1i[m,(x 2 k+l )] + £ Oi/WfG^+O + Ox.*+i»*+i 

t 

> /i[m,(:rl + ,)] + Z) Oi^,-(a;Ui) + ai.* + i*i + i - ( 3 ) 
»=i 

which is a contradiction since the quantity on each side of this inequality 
is equal to b t . If m,(xl +1 ) - m £ (^ +1 ) < then, 

/«KCr£ + i)] - fi[w.(An)] < 0. 
By applying Lemma 1 again, as above, one arrives again at (3) with 
> replaced by <. This is also a contradiction. Thus, we must have 

| xl +l - x\ +1 | ^ | mi(x 2 h+l ) - m t (x\+i) |, 

and hence (2) is proved 
Now, consider the function 



Let xi + i , »ILi e #* with x\ +l < x 2 k+1 . Then. 

lb 

Gtjfc+i.ifc+i ^ Z^ I a k + \,i I 
j-i 

implies 

- a* + i.* + , I x t 2 +1 - *i + i I ^ Z (I *«.i I-I **'+' ~ *2+' D- 
But, using (2), 

a 4+I . k+ ,(^ +1 - x\ +l ) ^ Z | a t+1 .,[m,-(xL0 - m,-(xl +1 )] I 
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^ — £ afc+i.f[m*(a£+i) - m,-(xi + i)], 
i -i 

which implies 

1 = 1 J - 1 

That is, the function (4) is monotone increasing. Clearly (4) is con- 
tinuous. It follows, therefore, that if f k+1 is a strictly monotone in- 
creasing function mapping E 1 onto E 1 , then so is the function 

* 

]k+\(%k+i) + 2-i fl *+i.i m i( I i+i) T flfc + i.fc + i3J* + i • 
i-i 

Thus, there exists a unique solution of the equation 

k 

If x° k+1 denotes this solution then 

m,(i2+i) 

nikixUi) 

n 

. X k + \ 

is the (unique) solution of 

F k+l (x) + A k+1 x = B k+l . 
Thus, we have proved that there exists a unique solution of (1). □ 

IV. BOUNDS ON THE SOLUTION 

Having established the existence and uniqueness of a solution of 
(1) a natural question to arise is: What can one say about the loca- 
tion of this solution? It turns out that we can say quite a bit (again 
assuming that conditions (i) and (ii) of Section I are satisfied). One can, 
in fact, with little effort (compared with the effort required, in general, 
to actually compute the solution) determine a finite region R in E n , 
in which the solution must lie. This region is the cartesian product of 
finite intervals /, C E 1 , for i = 1, • • • , n, each of which has the property 
that if 



1762 THE BELL SYSTEM TECHNICAL JOURNAL, OCTOBER 1968 



X n 

is the solution of (1) then xl e I, and, as 

£ I a,, I - 0, 

;=1 
)Vi 

the length of /<, J(A) -» 0. Thus, when the off-diagonal elements of 
A are small, the region R will also be small. 

In many applications it may be sufficient to know only that there 
exists a unique solution of (1) and to know the region R in which it 
must lie. If, however, one actually does want to compute the solution 
by some iterative technique, the knowledge of R should be useful in 
determining a starting point for the iteration. In fact, it will be 
shown that if the point x* is the solution of 

F(x) + diag [a„, • • • , a nn ] x = B, (5) 

then x* is also in R and thus might be a reasonable starting point 
for an iterative computation of x°. 

The computation of bounds for the solution of (1) proceeds in two 
steps. First, one solves each of the equations 

Ufa) = b it for i = 1, ••• ,n. (6) 

Letting a t denote the solutions of (6) , and denning 



a = max 



a,- 



\i = 1, ••• ,n\, 



B' = 



2 I «i» 






S I a m 



i-i 

C jVn 



one then solves each of the equations 

F(x) + diag [a n , • ■ • , a nn ] x = B — aB' , 
F(x) + diag [a„, • • • , a nn ] x = B + olB'. 



(7a) 
(7b) 
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V = 



V\ 



tin. 



and £ = 



respectively, one has R = I t X • • • X i», where 

h = [Vi . €<L for i = 1, ■■ ■ ,n. 

It is clear from the fact that each component of the vector aB' is a 
nonnegative number and from the monotone nature of the left-hand 
sides of (7) that x*, the solution of (5), is (as claimed) always in R. 
It is also clear that, for i = 1, • • • , n, as 



S I «« 



0, 



the ith components of both B — aB' and 5 + aB' approach b t , and 
hence 77,- — * a;* and £, — » z* . Thus, l(J { ) — » 0. We now prove that the 
solution of (1) is in R. 

Theorem 2: If R is constructed as described above, then the solution of 
(1) is contained in R whenever conditions (i) and (ii) of Section I are 
satisfied. 

Proof: Let x° be the solution of (1) and let k e {1, ••• , »} be chosen 
such that I x° k | = max{| x° { \: i = 1, • • • , n). Then, by Lemma 1, if 

xl > 0) S a H x °j = an d hence, 

- fM + £ a w *J - b k ^ f k (xl) - b k 
1=1 

or hixDl^ b k . 

Thus, because of the monotonicity of f k , 

\ x° k \ = x° k ^ a k ^ a, 
and hence | x° t | ^ a for i = 1, ■ • ■ , n. Similarly, by Lemma 1, if a;J < 0, 

2j 0**3° ^ and hence, 
1-1 

h(x° k ) ^ b k , 
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and thus 

| X° k I = -xl =g -a k ^ a, 

and hence \x° t \ ^ a, for i = 1, ■ • • , w. Thus, in any case, | x° \ ^ a, 
for i = 1, ■ ■ • , n. 

Now, for all x with | z,- | ^ a for j = 1, • • •, n, and for each 

i e (1, ••• , raj we have, 



« Z I «./ I = Ti (I a " I a ) = Si a .r*/ I. 

; = l ; = l i - 1 

which implies 

a 2 | a,, | ^ £ a*/*/ , 






and 



Thus, 



71 ** 

-a 2 I a « I = ^ a " X > • 

,Vi ifi 



a u Xi - a £ | a./ll = Z) a <. :K ; = <*«f*i + a S I a --» I- 

1=1 i-i ' =1 

In particular, for x = rc°, we have 

/,(z°) + aux°< - « E I fl« I ^ &* ^ f<(**) + «..^ + « E I «« I- 
,=i .;=i 

,»-.• i"» 

Comparing this result with (7) we have, as a consequence of the 
monotonicity of the functions on the left-hand sides of (7) , 

Vi ^ x°i ^ £. , for t - 1, • • • , n. 

Hence, x° t R. D 

Since in the above proof it was shown that | x° t | ^ a for i — 1, • • ■ , n 
it might seem to some readers that the intervals /,- might be reduced 
in length if we simply define them to be: /,- = [—a, a] Pi [77,- , £,]. This, 
however, is unnecessary since it is easily shown that —a £ i)i £ ti S <*, 
for i = 1, • • • , n. 

V. EXAMPLE 

We now give an example of the use of the above method for the 
computation of solution bounds. Consider the equation 
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M*i) 


+ 


5 4 


Xi 


= 


15 


, 




U(x 2 ). 




_-3 4_ 


X 2j 




13 




where j\ and / 2 are defined by 




/,(*.) = ( ^ ~ 2 


x, ^ 


LAs, - 1 


x, < 0, 


and 






x 2 + 9, 


x 2 ^ 3 


U(x 2 ) = "J 


4x 2 , 


-3 < x 2 < 3 








x 2 - 9, 




x 2 ^ 




3. 



Figure 1 shows the graphs of /i and / 2 . Since we know that the region 
R will be small if the off-diagonal terms of A are small enough, we have 
intentionally chosen an example in which these terms are rather large. 
The computation of a by solving (6) may be done by inspection for 
this example. One finds that 4"' = 17 implies that «i is slightly greater 
than 2, and since a 2 = 4 we have a = 4. Using this result in (7) one 
readily computes 



v = 




0.125 



s £ 



2.23 
3.2 



f.(x,) 



f 2 (i 2 ) 



2 +• 

-/I *X 




Fig. 1 — The nonlinear functions /i and / a for the example. 
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Actually, it is easily verified that the solution of this example is £° = . , 

VI. COMPUTATION OF THE SOLUTION 

For i = I, • ■ ■ , nwe denote by /'(#,) the right-hand derivative of/, at 
the point x f t E l . For each x t E n we denote by F'(x) the following matrix: 

F'{x) - diag WW, • • • , f n (x n )). 

It is easy to prove that if F satisfies condition (i) of Section I then 
F'(x) exists for all x t E n . Also, it is clear that each element of the main 
diagonal of F'(x) is nonnegative for all x c E n . (Each element is in 
fact positive if, in addition, F satisfies condition (in) of Section I.) 
Finally, we note that 



F~\y) - 



/:'(</,) 



fn\Vn) 



is defined for all y e E n , assuming again that F satisfies condition (i) of 
Section /. 

The following two iteration schemes are proposed for the computation 
of the solution of (1): 

Scheme 1 : For given x 1 t E n the sequence x 1 , x 2 , x 3 , • • • of points in E n is 
constructed by use of the formula 

x k+1 = [F'(x k ) + A]-\B - F(x k ) + F'(x k )x k ). (8) 

Scheme 2: For given x 1 c E n the sequence x 1 , x 2 , x 3 , • • • of points in FT is 
constructed by use of the formula 

x * + i = [F /(/p-»(/)) + AYHB - y" + F'iF-^y^F-^y")), (9) 

where y k = —Ax" + B. 

In order to explain the origin of (8) and (9) we make the following 
observations: If for i = 1, • • • , n (x] , y k ) is a given point in E 2 , and 
if we draw the graph of each of the functions /, , then each of the points 
in the sets { (x k , f { (x k )) : i = 1, • • • , n) and { (/7 1 (y k ), y k ) : i = 1, • • • , n) 
lies on the graph of the corresponding function /, . Suppose we now 
replace (approximate) each /< by the straight fine which is tangent to 
it at the corresponding point in one of the above sets.* Choosing the 



♦Our definition of tangent coincides with the usual one, except that the right- 
hand derivative is used at those points where the derivative fails to exist. 
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first set of points we approximate F by 

F(x) = F'(x h )x + F(x k ) - F'(x k )x k . 

Choosing the second set gives 

F(x) m F'(F-\y k ))x + y - F' (F~ l (y k ))F-\y k ) . 

If we now define y k = — Ax k + B and compute the solution of the 
equation 

F{x) + Ax = B 
and call it x k+1 , we obtain (8). Calling x k + x the solution of 

F(x) + Ax = B, 

yields (9). 

The above remarks have a very meaningful interpretation for problems 
arising from nonlinear electrical networks of the type described in 
Section I. Iteration Scheme 1 implements the following procedure: 
Given the vector x k of port voltages for the linear n-port, replace each 
two-terminal nonlinear device with a linear TheVenin's "equivalent" 
circuit whose V vs I curve is a straight line, tangent to the given curve 
at the point (x k , /,(:r*)). Compute the port voltages in the resulting 
linear network to obtain x +1 . 

Iteration Scheme 2 has a similar interpretation; this time, how- 
ever, the vector of port currents, y k = —Ax k + B, is used to determine 
the linear equivalent circuit replacing the nonlinear devices at each 
step. 

In view of the above remarks it is apparent that if one has some 
facility for solving linear network problems (a computer program, for 
example) then it might easily be adapted to solve many nonlinear 
problems as well. 

We finally remark that the use of the first iteration scheme is, in 
essence, the same as using the Newton-Raphson technique to compute 
the root of (1). 

We now prove a theorem which specifies conditions which are suf- 
ficient to ensure convergence of each of the above iteration schemes. 
We emphasize, however, that these iteration schemes will converge 
for many problems in which the conditions of the theorem are not 
satisfied — especially if a good enough starting point is provided. 

In the following we denote the origin in E n by 6 and, for the points 
x, y t E n , the notation x ^ y means x, ^ y> for i = 1, • • • , n. The 
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relations x < y, x *z y, x > y are defined similarly. We also make use 
of the concept of a matrix of monotone kind. 6 The matrix A is said to 
be of monotone kind if x z E n , Ax ^ 6 => x ^ 6. It is easy to show that A 
is of monotone kind if and only if A' 1 contains only nonnegative 
elements. It is also easy to show that if A is of monotone kind and 
x, y t E n with Ax ^ y, then x ^ A~*y. Ref. 6 shows that if the strict 
inequality > holds in condition (ii) of Section I, then conditions (it) and 
(iv) are sufficient to ensure that A is of monotone kind. 

Theorem S: For an arbitrary starting point x 1 , both of the above 
iteration schemes will converge to the solution of (1) if conditions 
(i) through (iv) of Section I are satisfied. 

Proof : We give here only the proof for the second iteration scheme, 
assuming that all of the functions / { are convex. The other three 
cases are quite similar and it will be apparent to the reader how 
this proof may easily be modified to take care of them.* 

We first remark that the iteration scheme is well defined. The fact 
that for every y e E", F'(F~ 1 (y k )) is a diagonal matrix containing all 
positive numbers on the main diagonal, and the fact that A satisfies 
conditions (ii) and (iv) of Section I, assures us that the matrix 
[F'(F~ 1 (y k )) + A] is nonsingular (it is, in fact, of monotone kind — see 
Ref. 6, p. 376). 

Let x 1 be an arbitrary point in E n . Then, since for i = 1, • • • , n 
and k = 2, 3, 4, • • • each of the points (x k , y k ) lies on some straight 
line, tangent to the corresponding function /, , and since each /,• is strictly 
monotone increasing and convex, we have that F~*(y ) ^ x for k = 
2, 3, 4, • • • .We now show that F" l (y k ) ^ x k implies that x k+1 ^ x k . 
Obviously, 

F'(F-\y k ))(x k - F-\if)) ^ d. 
But, by definition, Ax k + y' c — B = 8; hence, 

F'(F-\y k ))(x k - F~\y k )) + Ax k + y" - B £ 6, 
which implies 

W(F x tf)) + Mx k ^B-y k + F'(F-\y k ))F-\y k ). 
But then, since [F'(F _1 («/ fc ) ) + A] is a matrix of monotone kind, 



* After this manuscript had been completed, the author became aware of 
J. S. Vandergraft's paper (Ref. 7). With a certain amount of reformulation, the 
(monotone) convergence of the first iteration scheme, when all ft are convex, 
can be shown to follow, in essence, from his Theorem 5.1. 
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x k ^ [F'(F"V)) + AYUB - if + F'(F- I (7/ i ))F- 1 (7/ t )), 

or, a; ^ x +1 . Thus, the sequence re 2 , x 3 , x 4 , • • • has the property 

re 2 ^ z 3 ^ x 4 ^ 

We now show that for k = 2, 3, 4, • • • , re* ^ a; , where .t° is the solu- 
tion of (1). For each x k , k = 2, 3, 4, • • • , there is some point p t E" 
(p = F _ V -1 )) such that 

^x* - £ = F'(p)p - F'(p)x k - F(p). (10) 

Furthermore, from the convexity of each /,-, it is clear that for every 
pair of points q 1 , q 2 e E", 

F(q l ) ^ F(q 2 ) + F'foW - Q 2 )- 
In particular, 

F(x°) ^ F(p) + F'(p)(x° - p). 
Hence, 

F'{p)p - F(p) + F(x°) ^ F'(p)x° 
which implies 

F'(p)( v - x k ) - F(p) + F(x°) ^ F'(p)(x° - a:*). 
Using (10) we have, therefore, 

Ax" - B -f F(x°) ^ F'(p)(x° - x k ). 
But, F(x°) = - Ax° + B, hence 

A(x k - x°) ^ F'(p)(x° - x k ) 
or, 

[F'(p) + A){x k - x°) ^ 6. 

But then, since [F'(p) + A] is of monotone kind, x k — x° ^ 6, or x k ^ z°. 
Thus, we have shown that each sequence x] \ , x] , x\ , • • • is a bounded 
monotone sequence and hence the sequence x 2 , x 3 , x 4 , • • • converges 
to some point x* in E n . We now prove that x* = x°; that is, we show 
that x* satisfies (1). 

Let y* = -Ax* + B. Then, as k -> «>, 3* -> x * and ?/* -+ ?/*. Thus, 
^ _1 (2/*) — ► F~\y*) and each element of the matrix F'(F _1 (/)) approaches 
the corresponding element of F'(F~ '(?/*)). Now, from (9), we have 

Ax 1 " 1 + F'(F-V))* i+1 = Ax k + F'(F- , ( 2 / i ))F- I ( 2/ *) 
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which implies 

nrWxrW - ** +1 ) - A (x k+l - x k ) 

and hence 

F'(F-V))(F-V) - x* = A(x k+l - x k ) - F'(F-\y k ))(x* - x k+l ). 

But as k -> co, ( x k+1 - x k ) ~-> and hence A(x k+1 - x k ) -> 5; also, 
(x* - x k+1 ) -» 0and hence F'(F-\y k ))(x* - x k+1 ) -» F'(F-\y*))6 = 6. 
Thus, as fc — > co , 



which implies 



or 



and therefore 






Hence, ?/* = FOc*) , and thus, 

F(x*) + Ax* = 5. 

Thus, the iteration converges to the solution of (1). □ 

Although Theorem 3 states that both of our iteration schemes will 
converge for the same class of problems, only one of the schemes 
might converge for some problems for which all of the conditions 
(i) through (iv) of Section I are not satisfied. Also, for some problems 
a prior knowledge of the region in which the solution lies might dictate 
the choice of one iteration scheme over the other. For example, if it 
is known that some of the functions /< are quite steep in the neighbor- 
hood of the solution then perhaps F' 1 may be evaluated in this region 
more accurately than F. In this case Scheme 2 might be preferred 
to Scheme 1. 

VII. SPEED OF CONVERGENCE 

Section II mentions that in certain situations our iteration schemes 
may converge to the solution of (1) more rapidly than those based 
upon the contraction-mapping fixed point theorem. To illustrate this 
property we have chosen to compare the rate of convergence of 
Sandberg's iteration scheme to that of our schemes. 3 
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If we define the operator G mapping E n into E n by 

G(x) = F(x) + Ax, 

then, as a special case of Sandberg's Theorem I, we have the result: 
If there are positive constants k\ and k 2 such that 

(G(x) - G(y), x -y)^k x \\x-y || 2 , (11) 

and 

\\0\x) -G(y)\\*£ ^llx-^H 2 , (12) 

for all x, y e E n , then there is a unique solution of (1) and the solution 
is given by lim x k , where x 1 is an arbitrary point in E", and 



X 



ft+l 



= |i [B - G(x k )} + x\ 



for k = 1, 2, 3, • • • . The proof of this theorem consists of showing 
that the mapping 

H{x) = jr[B- G(x)} + x 

is a contraction. 

It is interesting to observe that if the inequalities (11) and (12) 
are satisfied then positive constants k 3 and fc 4 exist, such that 

(G(x) - G(y), x - y) ^ k 3 \\ x - y \\ 2 , (13) 

and 

\\G{x)-G{y)\\ 2 ^k i \\x-y\\\ (14) 

for all x, y e E n . In fact, a simple application of the Schwarz inequality 
to (11) and (12) yields (13) and (14) with k 3 = (fc 2 )* and /c 4 = k\ . 
Now (13) and (14) are of the same form as (11) and (12), except that 
the inequalities are reversed. Thus, if one uses (13) and (14) in the 
proof of Sandberg's theorem, reversing all inequalities, one obtains: 

\\H(x) - H(y)\\ 2 ^ K\\x - y\\\ 
where, 

K = 1 - 2(k 2 /k 2 ) i + (kl/k 2 ) 2 . 

It is readily seen that if 4k 2 < k 2 , then K is positive. If we let x° denote 
the solution of (1), and hence H(x°) = x°, we have, for k = 1, 2, • • ■ , 

II x k+1 - x° || 2 = || H(x k ) - H(x°) || 2 ^ K \\x k - x° || 2 . 
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Thus, (K)* represents, in this case, a lower bound on the rate of con- 
vergence of the iteration scheme. It is true that (K) h is always in the 
interval (0,1), for indeed Sandberg has proved that the sequence 
x k does converge to x°. However, as /bj becomes small, and as k 2 becomes 
large, K approaches 1 and the sequence converges quite slowly. For 
(1) the largest value that may be used for k t and the smallest value that 
may be used for k 2 will many times be dictated by the positive constants 
which are bounds on the slopes of the functions /,• . If, for example, 
the slopes of the /,- become so large for large x { , and so small for large 
negative x { that one must choose k x = 10" l and k a = 10 2 , then one 
easily computes (K)* tt 0.99. Thus, no matter how close any iterate 
is to the solution, the next iterate will be no more than about one per- 
cent closer. 

It is of course true that Sandberg's iteration scheme is applicable 
to a much more general class of problems than we consider in this 
paper. If, however, for any problem to which it is applied, the con- 
stants k x and k 2 must be restricted such that fci/fc 2 is quite small, 
then the rate of convergence will always be adversely affected. In 
the Katzenelson-Seitelman application of Sandberg's iteration scheme, 
their "heuristic refinement" (see Ref. 2) attempts to overcome this 
difficulty. 

Although the classes of equations to which our iteration schemes 
and the Katzenelson-Seitelman algorithm may be applied are not 
identical, in those cases where both techniques may be used the ad- 
vantage that our schemes offer is now clear. From (8) and (9) one 
easily obtains 

^« - x ° = [F '{x k ) + A]-\F(x°) - F(x k ) - F'(x k )(x° - x k )), 
and 

x k+1 - x° = 

[F'(F- l (y k )) + A}- 1 (F(x°) - y k - F'(F~ V))(z° - F' l (/))), 

respectively. These equations show that H^ 1 — x° || will be small 
(even if || x k — x° || is rather large) so long as for i = 1, • • • , n, 



Xi — Xi 



for Scheme 1, or 
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for Scheme 2. That is, as soon as the fcth iterate comes close enough to 
the solution that each of the functions /,• is approximately linear, the 
rate of convergence of our iterations becomes quite rapid. In fact, the 
rate of convergence increases without bound as the iterates approach 
the solution. It is also clear that if each of the functions U is piece- 
wise linear then our iterations will converge in a finite number of 
steps. 

From the standpoint of computational efficiency it is, of course, 
the amount of time required to compute an approximate solution 
that is the major concern. For those problems to which both our 
iteration schemes and the Katzenelson-Seitelman algorithm may be 
applied, it can happen that our methods might still be slower than 
theirs even in the case when the convergence rate of our methods is 
faster. This can happen because, for some problems, the equation 
with which we are concerned may be of a higher order than theirs, 
and also because we must compute the inverse of a matrix at each 
iteration step. On the other hand, it is clear that for many problems, 
even from the standpoint of total computation time, our techniques 
will be more efficient. 

VIII. ACKNOWLEDGMENT 

The author is grateful to I. W. Sandberg for encouragement and 
many helpful conversations. 

REFERENCES 

1. Duffin, R. J., "Nonlinear Networks II," Bull. Amer. Math. Soc, 53 (October 

1947), pp. 963-971. 

2. Katzenelson, J. and Seitelman, L. H., "An Iterative Method for Solution of 

Networks of Nonlinear Monotone Resistors," IEEE Trans. Circuit Theory, 
CT-13, No. 3 (September 1966), pp. 317-323. 

3. Sandberg, I. W., "On the Properties of Some Systems That Distort Signals-I," 

B.S.T.J., 42, No. 5 (September 1963), pp. 2033-2046. 

4. Minty, G. J., "Two Theorems on Nonlinear Functional Equations in Hilbert 

Space," Bull. Amer. Math. Soc, 69, No. 5 (September 1963), pp. 691-692. 

5. Weinberg, L., Network Analysis and Synthesis, New York: McGraw-Hill, 

1962. 

6. Collatz, L., Functional Analysis and Numerical Mathematics (tr. from Ger- 

man), New York: Academic Press, 1966. 

7. Vandergraft, J. S., "Newton's Method for Convex Operators in Partially 

Ordered Spaces," SIAM J. Numerical Anal., 4, No. 3 (September 1967), 
pp. 406^32. 



